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ABSTRACT 

Mobile learning is a cognitively demanding application and more frequently the ubiquitous nature of mobile computing 
means that mobile devices are used in cognitively demanding environments. This paper examines the nature of this use of 
mobile devices from a Learning, Usability and Cognitive Load Theory perspective. It suggests scenarios where these 
fields interact and presents an experiment which determined that several sources of cognitive load can be measured 
simultaneously by the learner. The experiment also looked at the interaction between these cognitive load types and 
found that a distraction did not affect the performance or cognitive load associated with a learning task but it did affect 
the perception of the cognitive load associated with using the application interface. This paper concludes by suggesting 
ways in which mobile learning (and mobile computing) can benefit by developing cognitive load aware systems that 
could detect and change the difficulty of the learning task based on the cognitive state of the learner. 
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1. INTRODUCTION 

Current mobile device use is presenting previously unconsidered problems. Mobile devices are now 
ubiquitous. The size, portability, battery life and computational power of mobile devices suggest that they 
can be used for a diverse range of uses in an equally diverse range of environments. Usability, or HCI 
(Human Computer Interaction), is primarily concerned with “ease of use” and “learnability” (e.g. how easy a 
system is to learn to use) (Neilsen, 1994). Mobile devices are challenging this notion of Usability as these 
devices are used in new ways. Firstly, mobile devices are used in complex distracting environments and these 
distractions interfere with the user’s cognitive resources. Secondly, mobile devices are becoming advanced 
and powerful and this allows more sophisticated applications to be used on them. 

Such applications are already cognitively demanding but the use of mobile devices for learning also 
brings a much higher demand for cognitive resources. Couple this demand to situations when the mobile 
device is used in a distracting environment and this leads to scenarios where the user’s cognitive resources 
become overloaded or stretched. Cognitive Load Theory explains how the human mind interacts with 
instructional materials for learning and has several guidelines that are used to assist in the design of 
instructional material used for learning. These guidelines suggest methods to discourage extraneous cognitive 
processes and encourage germane cognitive processes. This paper will present the results of an experiment 
that demonstrate how mobile learning, Cognitive Load Theory and mobile usability interact. 


2. LITERATURE REVIEW 

Learning is considered to be the acquisition and development of memories and behaviors, including skills, 
knowledge, understanding, values, and wisdom. Specifically, from the perspective of the Information 
Processing Model (Broadbent, 1958, Neisser, 1967), it is the changes made to long term memory, usually 
schema creation or automation. Learning has traditionally taken places in learning institutions or schools. 
However, this may not be the best place for learning (Dewey, 1916, Bloom, 1964). In modern times, not only 
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is high literacy demanded of everyone but the ability to think and reflect is now a valued attribute in the 
workplace. "Knowing" has shifted from being able to remember and repeat information to being able to find 
and use it (Simon, 1996). The role of education now should be the development of intellectual tools and 
learning strategies needed to acquire that knowledge (Bransford et al., 1999). 

Mobile learning can address some of these challenges. Mobile learning is defined as “Learning with the 
aid of a Mobile device” (Deegan and Rothwell, 2010a). In this definition a mobile device is simply a 
computer that is not restricted to a specific stationary environment or location. Specifically, mobile devices 
can be used in multiple environments, anytime, anywhere. Deegan and Rothwell, (2010a), investigated 
mobile learning from a Usability perspective and put forward a classification of mobile learning applications 
based on Usability. Usability is defined by the ISO as “ the effectiveness, efficiency and satisfaction with 
which specified users can achieve specified goals in particular environments .” (ISO 9241, 1998). A usable 
system should be easy to use (utility) and easy to learn (learnability) (Nielsen, 1994). In recent years the 
notion of 'ease of use’ has been applied to Usability where the general philosophy is based on making things 
’easy’ for the user. This approach stemmed from work done by Miller (1956) which determined that human 
cognitive resources were limited. 

One of the primary concerns raised during the creation of Deegan and Roth well’s, (2010a), classification 
was the issues surrounding Cognitive Load. Cognitive Load is the demand for mental resources associated 
with processing information in working memory. Usability practitioners attempt to reduce the inputs that 
humans had to process; making things easier (less Cognitive Load) meant making things more effective, 
efficient and satisfying to use. It was thought that systems that demonstrated a lowering of Cognitive Load 
would ensure that users were not over-burdened by the system. Computing began to reduce the Cognitive 
Load in an attempt to remove the burden of choice and thought from the user (Chalmers, 2003). Computer 
software is now easier to use and it is in part due to this that computer users of today generally have low 
Cognitive Loads applied to them (Sharp et al., 2007). While Usability practitioners attempted to lower 
Cognitive load, pedagogues and Cognitive Load researchers were finding that an increase in Cognitive Load 
could be beneficial for learning. Research reported in Sweller, (1988), Sweller and Chandler, (1994), Sweller 
et al., (1998), and Paas et a., (2004) etc. argues that our Cognitive Load can and will vary depending on the 
characteristics of the task and those of the user. Cognitive Load Theory (CLT) was developed to help 
understand how humans learn effectively with specific regard to problem solving. CLT is defined as being 
“concerned with techniques for managing working memory load in order to facilitate the changes in long 
term memory associated with schema construction and automation" (Paas et al., 2004). CLT posits three 
types of Cognitive Load that affect learners: Intrinsic, Extraneous and Germane Cognitive Load (Bannert, 
2002, Sweller et al., 1998): Intrinsic Cognitive Load (ICL) is due to the inherent difficulty of the task. 
Extraneous Cognitive Load (ECL) is due to distractions not directly related to the task and Germane 
Cognitive Load (GCL) results in changes to our long term memory due to the task (typically this is schema 
creation, manipulation, and assimilation). This activity, that leads to changes to long term memory, is directly 
associated with learning (Sweller et al., 1998). 

CLT actively encourages designers to increase the load associated with this schema creation (i.e. the 
GCL) and to reduce ECL in order to best use the learner’s limited cognitive resources and thus improve 
performance. Usability practitioners’ focus on increasing the ease of use of applications is, in CLT 
terminology, actually a focus on the reduction of ECL. These two contradictory aspects affect Mobile 
learning; as one aspect attempts to lower Cognitive Load another seeks to increase it. Van Nimwegen (2008) 
refers directly to this paradox: he questions the assumption that indiscriminately increasing the ease of use 
necessarily results in better applications. He determines that for certain types of applications it is actually 
beneficial to make the users’ task activities more difficult because this results in deeper thought processes and 
thus increased learning performance. Hollender et al., (2010) says there is no CLT counterpart in HCI for 
fostering germane load, perhaps because the goal of HCI is to reduce Cognitive Load (Sharp et al., 2007). 
However, it is notoriously difficult to measure cognitive load. If cognitive load can be measured it is then 
extremely difficult (if not impossible) to differentiate between its constituents: ICL, GCL and ECL. There 
have been advances in measuring the load especially in relation to user centered educational design (Oviatt, 
2006), however, little work exists that can accurately measure the different types of load by discriminating 
between them. This is particularly interesting to Usability and specifically learning applications where one 
needs to determine what aspects of cognitive load are aiding or inhibiting learning i.e. which aspect of 
instructional design is affecting the ECL or GCL. 
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3. MOBILE LEARNING INTERFACES 

Without a doubt learning is a cognitively demanding activity. As a result of Deegan and Rothwell's, (2010a) 
work, the need to balance Cognitive Load for the mobile learner was identified as a serious concern. 
Subsequently (Deegan and Rothwell, 2010b) presented a model of external sources of Cognitive Load that 
are relevant to mobile learning (Figure 1). This model acknowledged that Cognitive Load comes from the 
intrinsic nature of the learning (itself) but also suggested other extraneous areas that can contribute Cognitive 
Load and affect learning. 



Figure 1. Sources of Cognitive Load 

At the top of the model is the actual learning and this represents a small portion of likely Cognitive Load 
distractions (ECL's), perhaps the material being learned itself causes a distraction e.g. presentation. The 
learning application interface refers to the application itself and how the user interacts with the application 
e.g. uses its controls etc. The system software itself can also contribute to sources of Cognitive Load, e.g. 
badly timed error messages etc. The device could also contribute to Cognitive Load if it is badly designed or 
too heavy etc. Finally, the environment can contribute to cognitive load, and this represents the largest 
category of possible sources of Cognitive Load distractions. By combining several methods of balancing 
Cognitive Load (Kirschner, 2002, Mayer and Moreno, 2003, Van Merrienboer et al . , 2002, Sweller et al., 
2011) instructional designers can ensure that the learner’s cognitive load is at the suitable level with regard to 
the material being learned. They can make certain that the ICL and GCL of the instructional material is 
appropriate and make efforts to reduce or eliminate ECL. This addresses the cognitive load at the top of the 
model, the actual learning. 

Usability professionals can make efforts to reduce likely sources of ECL with regard to the application 
interface itself. However it is unlikely that those associated with the creation of Mobile learning applications 
will also have complete autonomy to control the cognitive load that is inflicted by the System software or the 
actual hardware of the mobile device. Attewell (2005) suggested that the technology selection used in Mobile 
devices varied greatly and therefore the system software and device hardware is deemed beyond the range of 
control of the mobile learning application developer. 

Finally the application developers could make efforts to develop mobile learning context aware 
applications that are also sensitive to the environment. In this way the cognitive load of the mobile learning 
application can be altered to suit the cognitive load of the environment. This will be a crucial attribute of a 
mobile learning application as mobile learning takes place with cognitively demanding applications in 
cognitively demanding locations. 

An early research question was formed that asked how exactly does a cognitive load, inflicted from a 
distraction, affect a learners performance and is it possible for a learner to distinguish between several 
different sources of these cognitive loads. Specifically could a user distinguish between the cognitive load 
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associated with the learning content, the cognitive load associated with the application interface and the 
cognitive load associated with the environment (e.g. a distraction)? If a user could distinguish between these 
loads it may be possible for the application to distinguish between the loads also. Developers could then 
create cognitive load aware applications which could alter the cognitive load that the application inflicts (e.g. 
making the application easier when a distraction is detected and difficult when the distraction is removed). 


4. EXPERIMENT TO DISTINGUISH BETWEEN COGNITIVE LOADS 

Traditionally cognitive load based experiments would ask the participant to perform a learning task. This 
learning task would have a specific ICL, or difficulty, and it may have an associated ECL based on its design. 
The user then would impose a GCL while completing the task. A total measurement of cognitive load would 
be ascertained at this stage. Usually one of these loads would be theoretically (as they cannot be measured 
separately) manipulated e.g. a distraction, that inflicts an ECL, is added to the experiment conditions. Finally 
another overall level of cognitive load is measured and any change in overall load would be a reflection of 
the added distraction, yet what cognitive load this distraction affected would not be certain. 

This experiment design is different in that it will attempt to measure the “added" cognitive load, as well 
as the cognitive load associated with the learning task and the cognitive load associated with using the 
application interface. This experiment will determine if this added load actually effects the performance or 
cognitive load associated with the learning task or interface i.e. rather than adding an extraneous cognitive 
load and measuring the overall cognitive load, this experiment will attempt to add an extraneous cognitive 
load and separately measure it and its effect on the original tasks and loads. 

4.1 Method 

4.1.1 Participants 

104 participants were recruited from the online site Mechanical Turk 1 . Several recent studies have found 
Mechanical Turk a valid source of participant recruitment for behavioral and user studies (for a full overview 
please see; Kittur et al., 2008, Buhrmester et al., 2011, Paolacci et al., 2010, Callison-Burch, 2009). In fact, 
Buhrmester et al., (2011) and Ipeirotis, (2010) suggest that the representative demographic characteristics of 
Mechanical Turk may be at least as diverse and more representative than traditional college or online based 
recruitments. Each participant was paid $0.10 to complete the task which took approximately 4 minutes. No 
demographic information was obtained. All participants agreed to give their consent. Participants were not 
able to complete the experiment more than once. 

4.1.2 Materials 

Participants were expected to have their own computer and access to broadband / the internet via a computer. 
The learning task was a series of 10 mathematical problems in which the participant needed to add or subtract 
three “two" digit numbers. The answer will always be two digits and fall between 25 and 90. Cognitive load 
was measured using a 9 point Likert scale developed by Paas (1992). This scale is widely accepted as being a 
very accurate measure of cognitive load (Sweller et al., 2011). 

4.1.3 Procedure 

The added distraction was designed to bring the learner close to a state of cognitive overload. There were two 
levels to this distraction, easy (low cognitive load) and hard (high cognitive load). Referring back to Millers 
(1956) work which suggests that humans working memory is limited, it was decided that this distraction will 
be a memory task which will run simultaneously to the learning task. The actual distraction is unimportant as 
long as it inflicts a similar (for the purposes of this experiment) cognitive load on the participants. 

For the purposes of this experiment it was desirable to investigate the load associated with the application 
interface and not the device itself, e.g. the difference between using a mouse and keyboard and using gesture 
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based interaction to interact with the application etc. For this reason two versions of the learning task were 
shown to the participants. A web based version and a web based mobile phone simulator version (Figure 2). 
The only difference being the application interface i.e. how the information is presented and accessed. 



Figure 2. Mobile Simulator desktop app and desktop app 

The distraction required the learners to remember how many apples were displayed at the top of various 
pages (see figure 2) throughout the primary task of completing the mathematical calculations. The low 
distraction was to remember seven apples over the ten mathematical tasks and the high distraction was to 
remember seventy apples over the ten mathematical tasks. The participants were randomly split into 4 
groups. Group 1 completed the mathematical task with the desktop interface and low levels of distraction. 
Group 2 completed the mathematical task with the desktop interface and high levels of distraction. Group 3 
completed the mathematical task with the mobile interface and low levels of distraction. Group 4 completed 
the mathematical task with the mobile interface and high levels of distraction. In addition to measuring the 
cognitive load of the mathematical task and cognitive load of the distraction, the cognitive load of the 
application interface was also measured. 

Experiment hypothesis are as follows. HI: The learner can measure several cognitive loads 
simultaneously, including loads that the mathematical task, the device interface and the distraction inflict. 
H2: The distraction will affect the mathematical task. H3: The cognitive load associated with using the 
mobile interface will be higher than for the desktop interface. 

4.2 Results 

The analysis focused on the mental effort and performance associated with the mathematical learning task 
(taskl) and distraction (task2) the mental effort associated with using the interface. 54 participants completed 
the task with the easy distraction (29 with the desktop interface and 25 with the mobile interface). And 54 
completed the task with the difficult distraction (27 with the desktop interface and 27 with the mobile 
interface). Difficulty and interface type with the independent variables with two levels each, respectively. 
The dependent variables were taskl effort rating, task2 effort rating, interface effort rating, taskl 
performance, task2 performance. The data was normally distributed, assumptions of variance-covariance and 
homogeneity of variance were met. As such a two way MANOVA was used to analyze the data. 

Using Pillai's trace, there was no combined effect of difficulty and interface on accuracy of taskl or task 
2, or the mental effort ratings of task 1, task 2 or the interface, V=.059, F(5,100)=1.247, p=.293, observed 
power of .43. However the MANOVA for difficulty alone, V=.131, F(5,100)=3.026, p=.014, showed that the 
difficulty of the task had a significant effect on the dependent variables with an observer power of 0.85. Also 
the MANOVA for the interface type alone V=.064, F(5,100)=1.37, p=.242 showed that the interface type did 
not have a significant effect on the dependent variables, with an observed power of 0.46. 

Follow up univariate analysis showed that difficulty had a significant effect on task 2 effort rating 
F(l,104)=4.991, p=.028 with an observed power of 0.60. This indicates that the difficulty of the distraction 
affected the distraction rating between the easy group (M=5.46, SD=2.67) and hard group (M=6.56, 
SD=2.24). Surprisingly the difficulty of the distraction also had a significant effect on the interface ratings 
and interface rating F(l,104)=5.58, p=.020, observed power 0.65, between the easy group (M=4.22, 
SD=2.56) and hard group (M=5.46, SD=2.55). 
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4.3 Discussion 

Hypothesis 1 was accepted, the learner was able to distinguish between several sources of cognitive load 
simultaneously. One of the main issues with Cognitive load Theory is that it is very difficult to differentiate 
between sources of ICL, GCL and ECL, where as an overall level of Cognitive Load can be measured 
(Sweller et al., 2011). In this experiment the learning task contains both ICL (17+12-3 has an inherent 
difficulty) and ECL (perhaps the presentation could have distracting elements). The distraction and the 
application interface are both sources of ECL. However they are simple measured by asking the learner to 
rate them. This is interesting as it suggests that it may actually be possible to differentiate between the 
various load types, although asking the learner to do so may inflict another ECL in itself. 

Hypothesis 2 was not accepted, the distraction did not significantly affect the performance of the primary 
task. This was unusual as current research suggests that a high secondary cognitive load can affect the 
primary performance. Sweller et al. (2011, p78) state “If the primary task imposes a heavy cognitive load, 
performance on the secondary task deteriorates. In contrast, a lower cognitive load on the primary task can 
result in improved performance on the secondary task. ” 

Unfortunately, a performance measurement for “using the application interface” could not be obtained so 
it is unclear whether the distraction made a difference in the performance associated with this action. As it 
seems there is a correlation between the mental effort associated with the distraction and the interface type, it 
is a distinct possibility that there could also be a relationship between the distraction and the performance 
associated with using the application interface. This possibly serves to highlight the importance of 
differentiating between the various cognitive loads so that individual relationships between load types can be 
determined and understood. Lor example, why did the distraction affect the cognitive load associated with the 
interface and not the learning task? 

Hypothesis 3 was also not accepted, there did not seem to be any significant effect of interface type on 
mental effort ratings or performance. This is good news for the mobile learning community. However, the 
actual conclusion may not be that clear. Lrom the graphs below (figure 3) it can be seen that the distraction 
was more pronounced (although not significantly different) for the mobile interface group than the desktop 
interface group. Interestingly, the univariate analysis for the effect of interface type on distraction ratings 
showed that there was a difference and while it was close to significant, it was not significant ( 
L(l,104)=3.709, p=.057 with an observed power of 0.48). It appears that the while difficulty had an effect on 
the interface rating (p=.20) the interface type had a near significant effect on the distraction rating. It is 
unclear why this relationship between distraction and interface type exists but it does suggest that a mobile 
learning application interface should possibly be tested under a state of high cognitive load in order to see 
these types of issues e.g. interface problems may not be apparent when the learner is not in a high state of 
cognitive load. Other univariate analysis for the effect of the interface type did not yield any significant or 
interesting results. 



Figure 3. Distraction and interface ratings 
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5. CONCLUSION 

This experiment demonstrated that individual ratings of cognitive load can be obtained instead of one overall 
level. Using this method various relationships between individual cognitive loads can be shown that would 
otherwise have gone unnoticed. In Cognitive Load Theory's search for a meaningful way to measure the 
relationship between individual loads this approach may be of benefit. Cognitive Load Theory is primarily 
concerned with the cognitive loads associated with a learning task and its instructional design but, as this 
research hopefully shows, cognitive load theory can be applied beyond learning itself to the application 
interface and distractions. And, possibly, when Cognitive Load Theory is applied in this way (to non learning 
elements), learning itself will benefit. 

Usability and Mobile Usability can benefit from these results by understanding the capacity for the user to 
differentiate between various sources of cognitive load. These individual sources of cognitive load may 
demonstrate that that perceived usability of an artifact may in fact vary based on the cognitive load of the 
user. The results of this experiment show that the mental effort needed to use an artifact changes based on the 
cognitive load of the user, e.g. available mental resources. This is beneficial to Usability in general but it 
may be critical to Mobile Usability specifically as the user moves between various environments and various 
sources of cognitive load. 

Learning is a cognitively demanding activity. This experiment measures other sources of cognitive load 
and demonstrates interactions between these loads. Mobile learning is concerned with 1) learning anytime, 
anywhere and 2) learning with the aid of mobile devices. Both of these activities inflict a great deal of 
cognitive load on the user, and should be considered when developing or planning mobile learning solutions. 

In fact, mobile learning applications should be developed with cognition aware systems that can interpret 
sources of cognitive load (e.g. camera, motion, sound sensors to detect environmental distractions etc.) that 
affect the learner and adjust the cognitive load (difficulty) of the instructional material to suit, so as not to 
overload the learner. Further similar experiments could help generate a more robust model of sources of 
cognitive load for mobile learning and this can be used to build such a system. 
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